System for non-disruptive insertion and removal of nodes in an ATM sonet ring

ABSTRACT

A system and a method of operating the system for non-disruptively inserting a node into the operations of an ATM ring uses the ring operations to update topology information and routing tables at the existing nodes to include therein the new node and one or more virtual paths associated with the new node. Basically, the node to be inserted establishes communications with a ring hub node over an established intra-ring management channel. The node and the ring hub node then exchange information over the management channel and the node performs its initializing routines in order that the node may operate to pass through traffic on the existing virtual paths. The new node next requests from the hub node the assignment of one or more virtual paths for directing traffic between that node and other nodes on the network. In response to the request, the hub node assigns one or more virtual paths to the requesting node and establishes a signaling channel to the node over one of the virtual paths. The hub node also notifies the other nodes of the assignment, and downloads to the new node routing tables that include the established connections over the existing virtual paths. The node may then participate in traffic shaping on existing virtual circuits, and also in call set up and call tear down operations over its assigned virtual paths in the same manner as the other nodes on the ring. A failed, or otherwise inoperative, node is removed from the ring without disrupting the traffic on the ring by essentially reversing the node insertion operations. After learning of the node failure, the hub node instructs the operative nodes to tear down the virtual path and associated virtual circuits that originate from or end at the failed node. Thereafter, the hub node directs the nodes to update their ring topology information to remove the failed node. In the meantime, the nodes continue to maintain the other virtual path and virtual circuit connections over the ring without disruption. When the failed node is later re-booted, the system follows the steps discussed above to re-insert the node, without disrupting the traffic on the ring.

FIELD OF INVENTION

The invention relates generally to asynchronous transfer mode (ATM)rings and, more particularly, to mechanisms for insertion and removal ofring nodes.

BACKGROUND OF THE INVENTION

Communication rings include a plurality of nodes that are interconnectedby a communication medium, such as fiber optic cable. In known prior ATMrings, the traffic on the ring is disrupted when a node is inserted intothe ring, that is, when the node is brought into the ring operations,and/or when a node is removed from the ring operations. In order for thenewly inserted node to operate in the ring, the existing nodes must beconfigured to make use of the virtual paths associated with the newnode. In one known ATM ring, the nodes on the ring must be re-configuredindividually and manually, that is, each must be reprogrammed.Accordingly, the traffic on the ring is disrupted as the operations ofeach of the respective ring nodes are interrupted so that the node canbe updated to make use of the new virtual paths.

The traffic on the ring is also disrupted each time a ring node has acatastrophic failure and must be removed from the ring operations. Whena node fails, the functioning ring nodes must again be reconfigured. Insuch systems, the traffic on the ring is also further disrupted when thefailed node is re-booted and re-inserted into the operations of thering. Ring traffic may also be disrupted when a node is temporarilyremoved from the ring for service or upgrading and thereafter re-bootedand re-inserted.

The reconfiguration of the nodes in the prior system must be performedunder the control of the system manager. Accordingly, the time toreconfigure each node, and thus, the time during which traffic on thering is disrupted at each node, and collectively over the ring, isrelatively long. Further, the time it takes to insert a node increasesas the number of nodes on the ring increases.

As the rings become larger and more traffic is sent over them, thelonger disruptions in the traffic are not as easily tolerated. This isparticularly the case when the ring is shared by multiple customers, andany disruption adversely affects more than the customer that is, forexample, joining the ring. Accordingly, what is needed is a system forinserting and removing a node without disrupting the traffic on the ATMring.

SUMMARY OF THE INVENTION

The invention is a system and a method of operating the system fornon-disruptively inserting a node into the operations of an ATM ring byfirst including the node as a virtual path pass through and then usingthe ring operations to update the routing tables at the new node and theexisting nodes to include therein the one or more virtual pathsassociated with the new node. The system non-disruptively removes a nodefrom the ring essentially by reversing the insertion operations. Thesystem and method are described below in terms of a SONET ring thathandles ATM traffic, but may be used in any ATM ring.

Basically, the node to be inserted establishes communications with aring hub node over an established intra-ring management channel. Thenode and the ring hub node then exchange information over the intra-ringmanagement channel and later over a virtual path that is assigned tothat node, to bring the new node into the ring operations in stageswithout disrupting the flow of ATM traffic over the existing virtualpath connections. For ease of understanding, we refer hereinafter to thenode that is inserted into the ring as the “new node.”

More specifically, once the new node is physically connected into thering it operates as an optical bypass. The new node then performs itsinitializing routines and, as necessary, communicates with the hub node,to allow the traffic on the existing virtual paths to pass through theprocessor on the new node. The new node next requests that the hub nodeassign one or more virtual paths for directing traffic between the newnode and the other nodes on the network, that is, between the new nodeand the other ring nodes and also the network nodes that are external tothe ring. In response to the request, the hub node assigns the virtualpath or paths to the new node and establishes a signaling channel to thenode over one of the virtual paths. The hub node also notifies the othernodes of the assignment, and downloads to the new node routing tablesthat include the established connections over the existing virtualpaths.

The new node is thus brought fully up-to-date. Accordingly, it mayparticipate in traffic shaping on existing virtual circuits, and in callset up and call tear down operations over its assigned virtual paths, asdiscussed in more detail below.

When a node fails, the system reverses the steps discussed above toremove the node from the ring without further disruption of the trafficon the ring. The hub node learns of the node failure when the failednode no longer communicates with the hub node, as discussed in moredetail below. The hub node then instructs the operative nodes to teardown the virtual paths and associated virtual circuits that originate orend at the failed node. The nodes continue to maintain the other virtualpath and virtual circuit connections over the ring, and the ATM trafficthat is directed to the operative nodes is thus not disrupted. When thefailed node is later re-booted, it is brought back into the ringoperations without disrupting the traffic on the ring using the stepsdiscussed above.

The operations of the current system are in contrast to known priorsystems in which the insertion of a new node and/or the failure andre-insertion of a previously operative node requires system-operatorcontrolled reconfiguration of the nodes on the ring.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention description below refers to the accompanying drawings, ofwhich:

FIG. 1 is a functional block diagram of a system constructed inaccordance with the invention;

FIG. 2 is the system of FIG. 1 with an additional ring node;

FIG. 3 is a flow chart of the operations of the system of FIGS. 1 and 2to insert a node; and

FIG. 4 is a flowchart of the operations of the system of FIGS. 1 and 2to removing a node.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

Referring now to FIG. 1, a system 10 that may be part of an ATM networkincludes a plurality of nodes 12 ₁, 12 ₂, . . . 12 _(i) that arereferred to collectively as nodes 12. One of the nodes, for example,node 12 ₁, operates as a ring hub node to control ring operations.

The nodes 12 are interconnected by a primary ring 14 and a secondaryring 16. The exemplary system 10 is a unidirectional SONET ring, withinformation in the form of ATM cells and frames transmitted in bothdirections over the primary and secondary rings. The system 10 may,however, be any type of ATM ring. We discuss below how a new node isnon-disruptively inserted into the ring and how a failed node isnon-disruptively removed from the ring. First, however, we discuss ingeneral the operations of the system 10.

1. Ring Operations in General

Each node 12 receives information over the rings 14 and 16 throughincoming ports 20 and sends information over the rings throughdestination ports 22. The nodes 12 also connect to other systems (notshown), including other networks or rings, or to various user terminals,through other incoming and destination ports 24 and 26.

The ATM cells are sent over the rings from source nodes to destinationnodes in virtual circuits. The system groups the virtual circuits intovirtual paths that include other virtual circuits with the same sourceand destination nodes.

Each virtual path is identified by a virtual path index (VPI), and eachvirtual circuit is similarly identified by a virtual circuit index(VCI). The VPI and VCI values are included in ATM cell headers, and areused by successive nodes to determine how to route the cells.

Intermediate nodes manage the cells on a virtual path basis. When anintermediate node receives cells over a virtual path on the primaryring, the node forwards the cells to succeeding nodes on the primaryring over the same virtual path. The intermediate node may, however, usethe virtual circuit information for traffic shaping in the virtual path,in a conventional manner. Similarly, when an intermediate node receivescells over a virtual path on the secondary ring the node forwards thecells to succeeding nodes on the secondary ring over the same virtualpath, and uses the virtual circuit information for traffic shaping.

Each destination node maintains routing tables (not shown) thatreference at least the associated virtual circuits over both the primaryand secondary rings and the destination ports to which the virtualcircuits are directed. The destination nodes select one ring interface,based on the quality of the traffic received over both rings, and directthe traffic from the selected interface to the appropriate ports.

The nodes 12 exchange ring management information with the hub node 12 ₁over a dedicated intra-ring management channel. In particular, the hubnode 12 ₁ and the other ring nodes 12 exchange ring topology informationover the intra-ring management channel. In the exemplary system 10, thetopology information includes updates to virtual path and virtualcircuit routing tables (not shown) that are maintained at each of thenodes, and also information relating to the responsiveness ornon-responsiveness of the various nodes. The ring nodes 12 also provideto the hub node 12 ₁ certain other information concerning the integrityof the primary and secondary rings by, for example, periodically sendingOAM cells to the hub node over the intra-ring management channel.

2. Ring Operations for the Insertion of a Node

Referring now also to FIG. 2, before a new node12 _(j) is inserted intothe system 10, that is, before the node participates in ring operations,the cabling and the node hardware are installed in a conventionalmanner. Accordingly, to install a new node 12 _(j) between, for example,the nodes 12 ₃ and 12 ₄, the cabling 15 for the primary ring 14 and thecabling 17 for the secondary ring 16 that connect nodes 12 ₃ and 12 ₄are unplugged from the ports 20 and 22 of the node 12 ₄. While thecabling is unplugged, the traffic between the nodes 12 ₃ and 12 ₄ isre-routed around the ring. The unplugged cabling is then plugged intothe appropriate incoming and destination ports of the new node 12 _(j)and additional cabling is run between the new node and the node 12 ₄.Electrical connections for the two rings are thus made through theconnection hardware 13 _(j) of the new node 12 _(j). The added cablingis depicted in the drawing by dotted lines.

Referring as well to FIG. 3, once the cabling is in place, the new node12 _(j) operates as an optical bypass, to physically pass ring trafficbetween the nodes 12 ₃ and 12 ₄ (step 300). The new node 12 _(j) mustthen be inserted into the ring operations, so that users connected tothe ring through the node 12 _(j) can send and receive information overthe ring.

The new node 12 _(j) performs its initialization routines, places allvirtual paths in pass through mode, and communicates with the hub node12 ₁ over the intra-ring management channel to exchange information thatallows a node processor in the new node 12 _(j) to control trafficpassing through the new node. In particular, the hub node and the newnode 12 _(j) exchange information that allows the node processor toprocess the ATM traffic on the existing assigned virtual paths throughthe ports 20 _(j) and 22 _(j) (steps 302, 304, 306). In the example, thenew node contains a list of a default range of virtual paths from whichall of the virtual paths on the ring are assigned, and the new nodeessentially configures itself, without hub node control, to operate as avirtual path pass through. If the new node did not contain such a list,the hub node would instead provide a list of the assigned virtual paths.

The new node 12 _(j) next sends a request to the hub node 12 ₁ over theintra-ring management channel, asking the hub node to assign to the newnode one or more virtual paths. The new node will later use thesevirtual paths to establish connections over virtual circuits to theother nodes in the system, both on and external to the ring (step 308).With the new node requesting its virtual path assignment, the hub nodeis freed from monitoring the passive activities of this node and theother new nodes that are being brought into the ring operations.Accordingly, the hub node may instead manage the dynamic operations onthe ring, such as call set up and tear down.

In response to the request from the new node, the hub node assigns oneor more virtual paths to the node. The hub node may then notify theother nodes of the new assignment (step 310). Alternatively, if the pathassignment is within the default range, the hub node may instead updatethe topology information to include the new node and rely on call set upoperations to update the routing tables when connections are establishedover the newly assigned virtual paths, as discussed below. The ringnodes 12 may instead learn of the new node through cells that areperiodically introduced to test ring continuity and/or to verify ringtopology.

The hub node next establishes a signaling channel over one of thevirtual paths assigned to the new node. The hub node and the new nodecan then exchange call set up and tear down information over thesignaling channel (step 312). The hub node next downloads to the newnode 12 _(j) over the management channel a set of up-to-date routingtables that include all of the existing virtual path assignments and/orthe associated virtual circuit connections (step 314). The routingtables are preferably the tables discussed in co-pending U.S. patentapplication Ser. No. 09/344,845 entitled RAPID CALL ESTABLISHMENT IN ATMRINGS which is assigned to a common assignee and is incorporated hereinby reference. In the exemplary system, the hub node also downloads anerror checking code, such as a check sum, that is used by the new node12 _(j) to verify that the routing tables were downloaded without error.

The node 12 _(j) may now fully participate in the ring operations. Morespecifically, the node 12 _(j) sends over the established signalingchannel requests to the hub node 12 ₁ to set up calls from associatedusers as virtual circuits over a newly assigned virtual path, and laterto tear down inactive connections over the assigned virtual path. Thiscall set up and tear down information is also sent by the hub node tothe other nodes on the ring, and these nodes then update their routingtables to include the connections. Based on instructions and updateinformation received by the new node at various times from the hub node,the node 12 _(j) updates its routing tables as new connections areestablished between the various other ring nodes, and as variousconnections are torn down. Also, the new node uses the virtual circuitinformation in the routing tables to shape traffic over the variousvirtual paths in a conventional manner.

3. Ring Operations for Removal of a Failed Node

Referring now to FIG. 4, when a node 12 fails, the system 10 removes thefailed node from the ring operations, without disrupting the operationsof the remaining ring nodes, essentially by reversing the steps for theinsertion of a node. The hub node learns of the failure of, for example,node 12 ₃, when the node 12 ₃ stops communicating with the hub node overthe signaling channel established between them. The hub node may alsolearn of the failure of the node 12 ₃ when the hub node does not, duringa predetermined maximum time interval, receive OAM cells or other cellsthat are periodically originated by the node 12 ₃. Alternatively, thehub node may learn of the failure when the node does not respondappropriately to the OAM cells or other test cells that the hub nodeoriginates and sends over the ring to test ring continuity and/or toverify ring topology (step 400).

Once the hub node has determined that the node 12 ₃ has failed, the hubnode instructs the other nodes on the ring to, as necessary, re-routetraffic through the operative nodes as part of a protection switchingoperation (step 402). If the electrical connections, that is, the cablesand associated hardware, through the failed node are functional, thefailed node 12 ₃ operates by default as an optical by-pass and the ringtraffic need not be re-routed.

The hub next instructs the operative ring nodes to tear down the virtualcircuits that originate from or are directed to the failed node (step404). In response, the nodes update their routing tables. The hub nodethen sends updated ring topology information to the remaining nodes, andinstructs them to remove from their routing tables the virtual pathsassociated with the failed node. At the same time, the remaining ringnodes 12 continue to send and receive ATM traffic associated with theoperative ring nodes over the ring in the usual manner, that is, overthe existing virtual paths and associated virtual circuits.

When the failed node 12 is later replaced or re-booted, the new or newlyoperative node is inserted into the ring operations in accordance withthe steps discussed above with reference to FIG. 3, and the traffic overthe ring is not disrupted. The same node insertion operations may alsobe followed after a node has been taken temporarily out of ringoperations for upgrading and/or service.

The system and method of operating the system described above allow ringtraffic to travel the ring without disruption while nodes are added orfailed nodes are removed from the ring. This is in contrast to theoperations of known prior ATM rings that require the nodes to beindividually reconfigured under the control of a system manager, inorder to add or remove a node from ring operations.

The foregoing description has been limited to a specific embodiment ofthis invention. It will be apparent, however, that variations andmodifications may be made to the invention, such as, for example, use ofany type of communication media between nodes, configuration of the ringas any type of dual-path communication network, with the attainment ofsome or all of its advantages. Therefore, it is the object of theappended claims to cover all such variations and modifications as comewithin the true spirit and scope of the invention.

1. A method for inserting a given node into ring operations of an ATMring, including: operating the given node as a bypass for ATM traffic onthe ring; operating the given node as a pass through for the ATM trafficon other existing virtual path connections on the ring before a virtualpath is established for the given node; assigning to the given node oneor more virtual paths to direct traffic to and from the given node overthe ring; communicating the virtual path assignment to other nodes onthe ring to establish the assigned virtual path on the ring; andproviding to the given node connection information for virtual paths andvirtual circuits on the ring.
 2. The method of claim 1 wherein the stepof providing connection information to the given node includes providingrouting tables to the given node.
 3. The method of claim 2 wherein thestep of providing connection information further includes providing theinformation from a hub node to the given node.
 4. The method of claim 3wherein the step of providing connection information further includes:providing an error checking code with the information, and at the givennode checking the information with the error checking code to determinethat the information is correct.
 5. The method of claim 1 wherein thestep of assigning the virtual path to the given node further include thegiven node requesting the assignment from a hub node, and the hub noderesponding to the request with the assignment.
 6. The method of claim 1wherein the step of communicating the virtual path assignment to othernodes includes updating routing tables maintained by the other nodes. 7.The method of claim 1 wherein the step of communicating the virtual pathassignment to other nodes includes providing to the other nodes call setup information for calls over the newly assigned virtual path.
 8. Themethod of claim 1 further including the steps of establishingconnections to and from the given node over the assigned virtual path;and tearing down connections over the assigned virtual path.
 9. Themethod of claim 8 wherein the step of communicating the virtual pathassignment to other nodes includes updating routing tables maintained bythe other nodes.
 10. The method of claim 9 further including updatingthe routing tables with call set up and tear down information associatedwith the one or more virtual paths assigned to the given node.
 11. Themethod of claim 1 further including the step of, at the given node,shaping traffic over the virtual circuits associated with theestablished connections on the ring.
 12. The method of claim 1, furtherincluding: establishing a connection for the given node with anintra-ring management channel; and exchanging pass through informationbetween the given node and a hub node on the ring via the intra-ringmanagement channel, the pass through information being used to operatethe given node as a pass through.
 13. A method for inserting a givennode into ring operations of an ATM ring and removing a failed node fromthe ring operations, the method including: operating the given node as abypass for ATM traffic on the ring; operating the given node as a passthrough for the ATM traffic on existing connections on the ring before avirtual path is established for the given node; assigning to the givennode one or more virtual paths to direct traffic to and from the givennode over the ring; communicating the virtual path assignment to othernodes on the ring to establish the assigned virtual path on the ring;and providing to the given node connection information for virtual pathsand virtual circuits on the ring; tearing down connections directed toand initiating from a failed node; and instructing non-failing nodes onthe ring to update ring topology information.
 14. The method of claim 13wherein the step of providing connection information to the given nodesincludes providing routing tables to the given node.
 15. The method ofclaim 14 wherein the step of providing connection information furtherincludes providing the information from a hub node to the given node.16. The method of claim 14 wherein the step of providing connectioninformation further includes providing a error checking node with theinformation, and at the given node checking the information with theerror checking code to determine that the information is correct. 17.The method of claim 13 wherein the step of assigning the virtual path tothe give node further includes requesting, at the given node, theassignment from a hub node, and responding to the request, at the hubnode, with the assignment.
 18. The method of claim 13 wherein the stepof communicating the virtual path assignment to other nodes includesupdating routing tables maintained by the other nodes.
 19. The method ofclaim 13, further including: detecting a failure with respect to thefailed node in response to the failed node failing to communicate with ahub node.
 20. The method of claim 13, wherein said connections includesat least one of virtual paths and virtual circuit connections initiatingfrom or destined to the failed node.
 21. A ring network for conductingasynchronous transfer mode (ATM) communications, comprising: a pluralityof ring nodes operably connected via a plurality of virtual paths, eachvirtual path being used to direct traffic from an initiating ring nodeto a destination ring node; and a ring hub node configured to instruct anewly-inserted ring node to operate as a pass through from ATM trafficvia the virtual paths until one or more new virtual paths areestablished for the newly-inserted ring node.
 22. The ring network ofclaim 21, wherein the ring hub node is further configured to assign thenew virtual paths to direct traffic to and from the newly-inserted ringnode.
 23. The ring network of claim 21, wherein the ring hub node isfurther configured to provide connection information to the ring nodes,the connection information corresponding to virtual paths and virtualcircuits on the ring network.
 24. The ring network of claim 21, whereinthe ring hub node is further configured to detect a failure of one ofthe ring nodes; and tear down, in response to the detected failure,connections on the ring network directed to or initiating from thefailed ring node.
 25. The ring network of claim 24, wherein the ring hubnode is configured to detect the failure in response to the failed ringnode failing to communicate with the ring hub node.
 26. The ring networkof claim 21, wherein the ring hub node is further configured to provideinstructions to the non-failing ring nodes to update ring topologyinformation at the non-failing ring nodes, the updated topologyinformation indicating that the failed ring node is removed from thering network.